Enable PHP FFE exposure system tests by leoromanovsky · Pull Request #7031 · DataDog/system-tests

leoromanovsky · 2026-05-28T13:07:35Z

Motivation

PHP FFE exposure logging now lives in the native dd-trace-php/runtime path and needs the shared FFE exposure system-tests enabled so regressions are caught outside local dogfooding. The M1 evaluation PR only proves remote-config evaluation; this stacked PR adds the next validation layer for exposure emission and cache behavior.

Planning/reference doc: https://docs.google.com/document/d/1NvMfTpZWLBlFmEFNjdnlMyeVpy5l7KD8qujGFco6w2w/edit?tab=t.0

Changes

This PR enables tests/ffe/test_exposures.py for PHP at v1.21.0-dev. It intentionally does not enable FFE evaluation-metric tests; those belong to a separate stacked PR for DataDog/dd-trace-php#3911 and #7033.

The PR is stacked on #7003, which adds the PHP FFE scaffold and evaluation-test enablement. The PHP behavior being validated here is implemented in DataDog/dd-trace-php#3910 and the sidecar delivery/cache support is implemented in DataDog/libdatadog#2026.

Decisions

The scope is deliberately one manifest activation. The shared exposure tests already cover the required product behavior: event generation, multiple RC files, malformed/empty RC handling, same-subject deduplication, different-subject emission, allocation/variant changes, doLog=false, missing flags, and empty targeting keys.

The local proof uses a matched PHP 8.2 NTS artifact and the php-fpm-8.2 weblog, which matches the PHP 8.2 target Bob recommended for FFE system-test iteration.

Related PRs

PHP runtime evaluation base: feat(ffe): add runtime-backed PHP feature flag evaluation dd-trace-php#3906
PHP exposure implementation: Add FFE exposure emission dd-trace-php#3910
System-tests PHP FFE scaffold/evaluations base: Enable PHP FFE evaluation system tests #7003
Sidecar delivery/cache: feat(sidecar): forward FFE exposures to EVP proxy libdatadog#2026

Validation

Local behavior validation before this conflict-resolution rebase (previous system-tests PR head a27a887bb), dd-trace-php exposure branch now pushed as 416749dd3, and libdatadog submodule 8be471fbc:

cd /Users/leo.romanovsky/go/src/github.com/DataDog/dd-trace-php-ffe-runtime-first
./tooling/bin/build-debug-artifact gnu-aarch64-8.2-nts \
  /Users/leo.romanovsky/go/src/github.com/DataDog/system-tests-pr7031/binaries

cd /Users/leo.romanovsky/go/src/github.com/DataDog/system-tests-pr7031
DOCKER_CONFIG=/tmp/system-tests-docker-config-nocredsstore \
SYSTEM_TEST_BUILD_TIMEOUT=1800 \
./build.sh --library php --weblog-variant php-fpm-8.2

DOCKER_CONFIG=/tmp/system-tests-docker-config-nocredsstore \
TEST_LIBRARY=php \
WEBLOG_VARIANT=php-fpm-8.2 \
./run.sh +l php FEATURE_FLAGGING_AND_EXPERIMENTATION tests/ffe/test_exposures.py

Result: 11 passed in 78.22s.

Post-rebase verification: branch head is now f54c036f4, stacked on the updated parent, enabling tests/ffe/test_exposures.py at v1.21.0-dev to match #7003. PATH=/opt/homebrew/opt/coreutils/libexec/gnubin:$PATH ./format.sh passed. The rebase only changed the manifest activation floor; the test behavior and local PHP artifact path above did not change.

MacOS arm64 local-run note: the published PHP 8.2 base image inspected locally was amd64-only, so the local validation first built datadog/system-tests:php-fpm-8.2.base-v1 for arm64 using ./utils/build/build_php_base_images.sh php-fpm-8_2. The temporary local base-image build tweak was reverted; this PR remains only the manifest activation.

datadog-prod-us1-4 · 2026-05-28T13:08:51Z

Tests

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 26 Pipeline jobs failed

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0 1

🔧 Fix in code (Fix with Cursor).
8 tests failed. Assertion Error: Expected to find flags 'test-flag-1' or 'test-flag-2' in exposure events, found: set().
🧪 1 Test failed
tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key.test_ffe_exp_5_missing_targeting_key[apache-mod-7.0] from system_tests_suite (Fix with Cursor)
AssertionError: EXP.5 FAILED: Expected exposure event for flag &#39;exp-5-missing-targeting-key-flag&#39; with subject.id = &#39;&#39;, but no matching event was found. Events received for this flag: []. The tracer must NOT skip exposures when targeting key is empty.
assert None is not None

self = &lt;tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key object at 0x7f785501ec60&gt;

    def test_ffe_exp_5_missing_targeting_key(self):
        &#34;&#34;&#34;EXP.5: Test that empty targeting key generates exposure with subject.id = &#39;&#39;.&#34;&#34;&#34;
        assert self.response.status_code == 200, f&#34;Flag evaluation failed: {self.response.text}&#34;
    
        result = json.loads(self.response.text)
...
Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0-zts 1

🔧 Fix in code (Fix with Cursor).
8 failed tests. AssertionError: Expected to find flag 'test-flag' with subject 'test-user' in exposure events.
🧪 1 Test failed
tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key.test_ffe_exp_5_missing_targeting_key[apache-mod-7.0-zts] from system_tests_suite (Fix with Cursor)
AssertionError: EXP.5 FAILED: Expected exposure event for flag &#39;exp-5-missing-targeting-key-flag&#39; with subject.id = &#39;&#39;, but no matching event was found. Events received for this flag: []. The tracer must NOT skip exposures when targeting key is empty.
assert None is not None

self = &lt;tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key object at 0x7f63f862c5c0&gt;

    def test_ffe_exp_5_missing_targeting_key(self):
        &#34;&#34;&#34;EXP.5: Test that empty targeting key generates exposure with subject.id = &#39;&#39;.&#34;&#34;&#34;
        assert self.response.status_code == 200, f&#34;Flag evaluation failed: {self.response.text}&#34;
    
        result = json.loads(self.response.text)
...
Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.1 1

🔧 Fix in code (Fix with Cursor).
8 failed tests related to missing exposure events for flags during evaluation. Expectation not met: expected flag exposure events to be found but none exists.
🧪 1 Test failed
tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key.test_ffe_exp_5_missing_targeting_key[apache-mod-7.1] from system_tests_suite (Fix with Cursor)
AssertionError: EXP.5 FAILED: Expected exposure event for flag &#39;exp-5-missing-targeting-key-flag&#39; with subject.id = &#39;&#39;, but no matching event was found. Events received for this flag: []. The tracer must NOT skip exposures when targeting key is empty.
assert None is not None

self = &lt;tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key object at 0x7fe512c86210&gt;

    def test_ffe_exp_5_missing_targeting_key(self):
        &#34;&#34;&#34;EXP.5: Test that empty targeting key generates exposure with subject.id = &#39;&#39;.&#34;&#34;&#34;
        assert self.response.status_code == 200, f&#34;Flag evaluation failed: {self.response.text}&#34;
    
        result = json.loads(self.response.text)
...
View all 26 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: f54c036 | Docs | Datadog PR Page | Give us feedback!}

github-actions · 2026-05-29T01:00:21Z

CODEOWNERS have been resolved as:

manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php

test(ffe): enable PHP evaluation system tests

224c241

This was referenced May 28, 2026

Enable PHP FFE evaluation metric system tests #7033

Draft

Add FFE evaluation metrics DataDog/dd-trace-php#3911

Open

Add FFE exposure emission DataDog/dd-trace-php#3910

Open

Merge branch 'main' into leo.romanovsky/pr-g-php-ffe-scaffold

8fbb3a5

leoromanovsky mentioned this pull request May 28, 2026

feat(sidecar): forward FFE exposures to EVP proxy DataDog/libdatadog#2026

Open

leoromanovsky added 8 commits May 28, 2026 11:38

Fix PHP FFE test activation versions

9bcf43d

Skip stale PHP FFE dev build

e2d4370

Bump PHP FFE tests to 1.21 dev

19b9c41

Remove PHP-specific FFE test branching

83bc16b

Use OF7 JSON fixture in FFE parametric tests

afd0d5f

Merge origin/main into PHP FFE tests

1d6ac67

Restore deterministic PHP FFE fixture startup

552010f

Enable PHP FFE exposure system tests

f54c036

leoromanovsky force-pushed the leo.romanovsky/pr-h-php-ffe-exposures branch from a27a887 to f54c036 Compare May 29, 2026 00:59

Base automatically changed from leo.romanovsky/pr-g-php-ffe-scaffold to main May 29, 2026 15:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable PHP FFE exposure system tests#7031

Enable PHP FFE exposure system tests#7031
leoromanovsky wants to merge 10 commits into
mainfrom
leo.romanovsky/pr-h-php-ffe-exposures

leoromanovsky commented May 28, 2026 •

edited

Loading

Uh oh!

datadog-prod-us1-4 Bot commented May 28, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

leoromanovsky commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Decisions

Related PRs

Validation

Uh oh!

datadog-prod-us1-4 Bot commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

github-actions Bot commented May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

leoromanovsky commented May 28, 2026 •

edited

Loading

datadog-prod-us1-4 Bot commented May 28, 2026 •

edited

Loading